Transit providers interconnect the networks that collectively form the Internet by providing the paths that allow one network to directly or indirectly exchange traffic with other networks. To provide a path, a transit provider typically operates a set of routers or network circuits that connect and allow for intercommunication between one network and another. A transit provider typically operates the routers along its paths, although the transit provider may sometimes leverage routers of other transit providers to complete a path, whereby relationships between the transit providers allow them to pass traffic on one another's routers or circuits. Some transit providers include NTT Communications, Tata Communications, Telefonica, TeliaSonera, and Verizon.
The term “path” can be defined to be any arbitrary set of routers or hops under control of a common transit provider through which a source operating within a first network can reach a destination operating within a second network. When different packets are sent from the source to the destination over a transit provider path, the packets can traverse different sets of routers or hops that are under control of the transit provider. The term “transit” refers to a specific path or a specific set of routers or hops under control of a common transit provider through which the source operating within the first network can reach the destination operating within the second network. In other words, when different packets are sent from the source to the destination over a specific transit, the packets traverse the same set or routers or hops. These terms will be used interchangeably in the following disclosure.
Network operators including, for example, content delivery networks (CDNs), cloud service providers, and other content providers, establish peering relationships with two or more transit providers so that they have multiple paths over which they can pass traffic to a destination. The multiple paths provide the content provider with failover options in the event that the path of one transit provider fails or performance along that path is degraded. For instance, if a content provider is unable to reach a destination over a first transit provider's path, the content provider can transition to a second transit provider's path in order to reach the destination through the alternate path.
Routers, by operation of the Border Gateway Protocol (BGP), control path selection for traffic that exits one network and is intended for another external network. Therefore, in standard network routing, path selection is obfuscated from and not available at the application level. Consequently, servers operating behind the router have no control over which of several available paths the router will pass server generated traffic.
A BGP compliant router performs path selection based on route advertisements it receives from neighboring BGP compliant routers. Although the BGP compliant router may be aware of different transit provider paths to a given destination, the BGP router usually selects the same path that it believes to be the most optimal to the destination based on the received advertisements. The BGP router may not deviate from the optimal path until that path fails or is no longer available.
There are several disadvantages to this standard BGP method of operation. First, BGP path selection can lead to unbalanced utilization of the available paths. One path can receive heavy traffic load, thereby causing that path to become congested and ultimately underperform relative to other unused and underutilized paths. Second, the BGP update messages do not advertise performance metrics so as to identify path performance from one endpoint to another. Instead, the BGP update messages identify the number of hops or routers separating a source network from a destination network. As a result, the BGP selected path, although containing fewer hops, may actually provide worse performance (e.g., greater latency, more packet loss, etc.) than another available path. The server sending the traffic has no control over the path selection and its content delivery performance can suffer as a result of BGP controlling the path selection at the network or router level rather than offering such controls at the application or server level. Third, BGP may be unable or slow to respond to certain failures. Since a BGP router receives advertisements from only its neighboring routers, it may take time for a router failure at one end of a path to propagate through to all other routers along the path. In the meantime, BGP routers can route traffic along the failing path with that traffic ultimately being undelivered to its intended destination. Stated differently, a BGP router does not know the full end-to-end connectivity of a path, and therefore, may be slow to respond to any problems occurring to routers that are not direct neighbors.
Some routers provide policy-based routing controls. The policy-based routing controls incorporate additional intelligence in the BGP router path selection determination. However, a network operator can configure policy-based controls only on the routers the network operator controls. Thus, the network operator may configure its router to send traffic to a first hop along a faster first path to a destination, however the router at the first hop may switch the traffic to a second hop along a slower or unresponsive second path, thereby defeating the ability of the network operator to control path selection. Another shortcoming is that the policy-based routing controls are only implemented in one direction, specifically when sending traffic from the network operator to a destination. Any traffic sent from the destination back to the network operator will not be subject to the same policy-based controls, as the network operator typically has no control over the routers at the destination end of the transit provider path.
As content providers and CDNs strive to optimize all aspects of content delivery, it is desirable for them to also gain control over path selection. Accordingly, there is a need to provide path selection control at the application or server level rather than at the network or router level. There is further a need to apply any path selection optimizations in a bidirectional manner. In summary, there is a need to override BGP without disabling BGP or router intercommunication via BGP.